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STEREOSCOPIC VIDEO OBSERVATION AND IMAGE MAGNIFICATION 

SYSTEM 



TECHNICAL FIELD 

The present invention relates generally to a stereoscopic observation 
system, and more particularly to a stereoscopic video observation and 
magnification system to be utilized in surgery and other medical applications. 

BACKGROUND ART 

At present surgeons performing surgical and other medical operations are 
obliged to utilize different observation methods across a set of different surgical 
sessions or even within a single session. Each observation method involves the 
use of different instruments and equipment. Although for some surgical 
operations unaided or natural observation is sufficient, many types of surgery 
demand magnification of the operating field view and enhancement of the 
surgeon's vision. Surgeons performing such surgical procedures are provided 
with a magnified view of the area upon which they operate by utilizing a series of 
optical devices having an associated range of magnification capabilities. The 
group of operative magnifying devices comprises various optical systems such as 
simple hand-held loupes, wearable binocular loupes with or without headlight, or, 
in the case of microsurgery, surgical microscopes. Typically, loupe-aided 
observation provides magnifications in the range of 2 to 3 (also larger 
magnification is available but the equipment is unwieldy and less frequently 
used) whereas the range of magnifications for a microscope is typically 8 or 
above. Utilizing diverse observation methods and related equipment is a distinct 
disadvantage of the present surgical observation technique. None of the existing 
techniques provide "see through his eyes" option or the capability of conveymg 
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the image seen by the surgeon to the eyes of the surgeon assistants and other 
experts present at the performance of a surgical procedure. 

Trends of modem surgery point towards the development of less invasive 
and minimally invasive surgery techniques, which provide less exposure and 
therefore require more magnifying and enhancing vision and visualization tools. 
Such less invasive surgery techniques include microsurgery needing a surgical 
microscope, endoscopic surgery using laparoscopes or endoscopes, computer 
assisted surgery, robotic and telesurgery implemented by using computer assisted 
systems associated in specific procedures to augmented reality techniques, and 
classical surgery procedures or direct minimal invasive surgery that involve 
minimal access to the operation field. 

Some of the surgeries mentioned above such as endoscopic surgery and 
computer assisted, robotic and telesurgery necessitate specific visualization 
systems, and may require, in addition, utilization of interfacing magnification and 
enhancing vision devices. Other methods of surgery such as microsurgery and 
classical surgery procedures involvmg the utilization of an operating room 
surgical microscope demand magnification and enhancement of the surgeon's 
vision. The treatment of remote organs and anatomical spaces, which are difficuh 
to expose and to reach during surgery as well as generic surgical steps such as 
microsurgical dissection, suturing techniques, free flap, artery and vein 
anastomosis, interposition vein graft, end to side anastomosis, nerve graft, 
maxillofacial microsurgery, vasovasostomy, implant organ surgery, fallopian tube 
anastomosis and the like, would substantially benefit from magnification and 3D 
online imaging. Modem surgical procedure demand upgraded conditions of 
visualization to improve the quality of surgery, reduce the length of the operation, 
the trauma for the patient and therefore reduce the complications rate and 
hospitalization length. 

An advanced visualization tool should be an integrated system comprising a 
real time imaging system with adjustable display, that will provide a 
three-dimensional high-resolution color image balanced and optimized 
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automatically and dynamically to comply with the changing conditions during the 
surgery, wide-range magnification tools, and readily utilized operator controls 
enabled to change some of the operative parameters according to the prevailing 
conditions. The system should provide the option of "see-through" to the surgeon 
during the surgical procedure. The system should also provide "see through his 
eyes" capability to the assistants and experts during the surgical procedure by 
suitably transmitting identical images to those displayed to the surgeon to the 
eyes of additional persons present during the procedure. Additionally the system 
should provide recording capability to enable repeated replay of the record of the 
procedure to the purpose of study, analysis, archiving, and the like. Prior art 
systems do not provide such capabilities. 

DISCLOSURE OF INVENTION 

One aspect of the present invention regards a method of displaying in real 
time, dynamic, variably magnified, optimized and stereoscopic images of a 
physical application space to a human operator to aid in observing and 
manipulating objects within that application space, the method contains 
generating dynamic images of the application space by image-sensing devices, 
encoding the images generated into electric signals, transmitting the signals 
representing the dynamic images to display devices situated in optical alignment 
with the eyes of an operator, decoding the signals representing the dynamic 
images and displaying the resulting reproduced images to the eyes of the 
operator. 

A second aspect of the present invention regards a method of providing a 

human operator with colored, high-definition, variably magnified, dynamic, 

stereoscopic, qualitatively optimized reproduced images of a physical operating 

environment to aid the operation during an observation and manipulation session 

associated with objects within the environment. The method contains viewing 

images of the session by means of video cameras, transforming the images mto 

video signals, sending the video signals via a set of suitable conduits to respective 

-3- 
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viewing screens situated between the operating environment and the eyes of flie 
operator, decoding the video signals representing the video images into 
reproduced images, and displaying the reproduced video images on the viewing 
screens. 

A third aspect of the present invention regards a stereoscopic observation 
and magnification system that contains two video cameras mechanically linked 
and embedded into a cameras module wearable by the operator, two display 
screens embedded into ahead mounted display unit wearable by the operator, an 
operator controller unit, an interface and processing unit, a suitable video circuit 
connecting the cameras, the display screens, via which the cameras transmit video 
signals representing the dynamic images of the operating environment, serial 
communication links and circuit via which image quality control parameter 
values of the cameras are sent to the interface and processing unit and to thew 
operator controller unit in order to provide monitoring and consequently 
balancing and qualitatively optimizing the reproduced images. 

Each and all the above aspects of the present invention provide improved 
quality of observation. 

Each and all the above aspects of the present invention provide improved 
quality of object manipulation, and reduced duration of the manipulation session. 

BRIEF DESCRIPTION OF THE DRAWTNGS 

The present invention will be understood and appreciated more fully 
from the following detailed description taken in conjunction with the drawings in 
which: 

Fig. 1 is a pictorial presentation of the main components of the system 
showing the functional locations of the components in relation to the user's body; 

Fig. 2 is a simplified block diagram showing the main components of the 
system interconnected in a functional manner, 

-4- 
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Fig. 3 is a graphic presentation of the geometric factors that define the 
stereoscopy-related spatial relationships among a pair of cameras and an observed 
object; 

5 

Fig. 4 is a graph of a qualitative performance of a stereoscopic system 
depending on the variation of the Inter Pupillary Distance in a given camera to 
object distance; 

10 

Fig, 5 is a schematic representation of the disposition of the components of 
the system; 

Fig. 6 is a functional block diagram showing the hardware and software 
15 components of the Interface and Processing Unit; 

Fig. 7 is high-level flow chart of the main operative function of the system; 

Fig. 8 is a high-level flow chart corresponding to the sequence of control 
20 activation on the Operator Controller unit; 

Fig. 9 is a high-level flow chart associated with the video level control using 
the readouts of the luminous data of the cameras; 

25 

Fig. 10 is a high-level flow chart of the focus control using the readouts of 
the cameras; 

Fig. 11 is a high-level flow chart associated with the adjustment of IPD and 
30 DA values using the prior reading of the focus; 

-5- 
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Fig. 12A, 12B, and 12C illustrate the mechanical structure of the Cameras 
Module including the motors and associated mechanical devices controlling the 
Inter Pupillary Distance (IPD) and the Convergence Angle (DA). 

BEST MODE FOR CARRYING OUT THF. n^NTTON AND TNDUSTRTAT. 
APPLICABILITY 

The present invention overcomes the disadvantages of the prior art by 
providing a novel versatile stereoscopic viewing system in order to aid at surgery 
and other medical treatment requiring magnification of details in the surgery 
space. 

The invention proposed is a computerized stereoscopic observation and 
magnification system for surgical purposes. This system combines picture quality 
of high-resolution video, flexibility of high power zoom and the ease of use 
achieved by the computerized automation. By using this powerful and efficient 
mechanism, the surgeon will be capable of working in a more effective and 
accurate manner thus it will be enabled to conduct difficult and complicated 
operations within a shorter period of time. The system provides for opthnal 
performance, and maximum automation in order to improve the surgeon's 
concentration and reduce the physical effort involved in a complicated and 
lengthy surgical procedure. This system provides a variable field of view, 
continuous control of the magnification from about 1 (like the unaided eye) to 
about 10 by using practically the same equipment and in the same session if 
required. The entire system is lightweight and has low power consumption, which 
makes available the option for emergency crews to use it as field equipment. 
Additionally the system provides a set of advanced features such as a 
"see-through" capability, a "see-though-his-eyes" option and a recording and 
playback capability. 
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Reference is now made to Fig 1, and Fig. 2 illustrating the main components 
of the system as designed to be utilized as an aid for the performance of a session 
involving close observation and manual or semi-manual manipulation of objects. 
It would be easily perceived that the user of the system could be one of several 
6 types of operatives functioning in diverse domains that involve the performance 
of manual or semi-manual manipulations of objects inside an optionally bounded 
physical application space which necessitate intense, concentrated observation of 
the manipulated objects therefore demands variable magnification of the observed 
view as long as the manipulation session is in progress. The disclosed 

10 embodiments are merely exemplary of the proposed invention. The details 
disclosed should not be interpreted as limiting but only as means contributing to 
the clearer understanding of the present invention. 

In the preferred embodiment of the present invention user 10 is a medical 
practitioner such as a surgeon utilizing the system as an aid to the performance of 

15 medical procedure such as surgery. User 10 preferably wears the main 
components of the system on or close to his body. Cameras Module (CM) 20 
comprises a pair of light-weight sensor devices specializing in image-reception, 
image-encoding and image-transmitting such as video cameras mounted on a 
supporting apparatus such as a headband, headset or helmet which is positioned 

20 on user 10 head. The headgear and the two video cameras are configured in such 
a manner, that temporarily simultaneous but spatially slightly different 
transmissions of appropriate digitally encoded electric signals, representing two 
monoscopic views, are provided to user 10 eyes thereby making available to user 
10 the option of observing the surgery space in a stereoscopic mode. The electric 

25 signals that represent the real-world images are received optically by the cameras 
and are transmitted via suitable conduits to the Interface and Processing Unit 
(IPU) 50 which is loosely attached to user lO's body. The attachment is achieved 
by the use of suitable means such as a belt-mounted device and the like. Interface 
and Processing unit (CPU) 50 processes the digital signals automatically and in 

30 accordance with user 10 manual instructions entered through Operator Controller 
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(OC) 40. The user enters instructions by manually activating various controls 
such as pushbuttons located on the upper surface of OC 40. OC 40 is shown 
attached by suitable means such as a strap or a Velco strip to the lower arm of 
user 10. The processed signals are transmitted via suitable cables to a Head 
Mounted Display (HMD) 30. Head Mounted Display 30 is situated in optical 
alignment between user 10 eyes and the surgery space being observed. Like all 
the other main components of the proposed system, HMD 30 is also wearable and 
in the preferred embodiment of the invention situated in front of the user 10 eyes 
in a similar manner to magnifying glasses. The signals transmitted to HMD 30 are 
decoded, structured into corresponding images and displayed on a pair of display 
devices such as Liquid Crystal Display (LCD) glasses and the like. 

It would be easily perceived by one skilled in the art that the strucmre, 
location and the arrangement of the described components could be different for 
other contemplated embodiments of the present invention. For example, in a 
different embodiment of the present invention Operator Controller unit could by 
utilized by activating the operative controls thereon by wrist, foot, touch or voice. 
Details disclosed herein are merely the basis for the following claims. 

The system described in Fig. 3 employs two image-sensing devices such as 
two video cameras 22 and 24 corresponding to the two eyes of an observer. The 
cameras 22 and 24 are separated from each other by the gap known as Inter 
Pupillary Distance (IPD) 25 and are situated at a distance 28 from the object 32. 
Each camera is aimed to the object 32 from a slightly different angle so the Unes 
of sight converge at the object 32. The lines of sight form a Convergence Angle 
(DA) 34. The picture is taken by each camera is displayed to the corresponding 
eye of the observer using the Head Mounted display HMD 30 of Fig. 1, The brain 
converts both pictures into three-dimensional scenery by virtue of its ability for 
depth perception. 
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Refeiring now to Fig. 4 where a qualitative performance of a stereoscopic 
system based on the variation in the value of IPD and the Converging Distance D 
is illustrated. Tests have been conducted to establish what is known as "the ideal 
3D system" for a given situation. When establishmg an optimal IPD and 
5 Converging Distance (D) the results of the tests show that there is a range of 
parameters that yield optimal performance. Referring to the ratio between the IPD 
and the distance of the object from the camera (D) as the main important measure, 
it is possible to discriminate between hypo stereoscopy 36 (too small IPD), 
optimal stereoscopy 38 and hyper stereoscopy 42 (too large IPD). In hypo 

10 stereoscopy the lack of depth perception is felt. Hyper stereoscopy yields 
confusion after a short time and fatigue. In order to optimize stereoscopic 
performance, the IPD is preferably re-adjusted according to the distance to the 
object. Unlike other 3D-video systems, the proposed system is capable of 
dynamically manipulating the IPD and Convergence Angle in order to achieve 

15 optical stereoscopy so that the depth effect is optimized without the undesired 
effects associated with loss of concentration or fatigue. 

The system employs two Charge Coupled Device (CCD) or Solid State 
Sensor cameras that are embedded into a Camera Module. For the cameras Sony 

20 Japan Model EVI 370DG or the like could be used. The cameras should be 
provided with an embedded DSP (Digital Signal Processor) in order to enable 
operation in fiilly automatic mode or to be controlled externally through a serial 
conmiunication port such as RS232Ai^SCA^ using Sony's special protocol. The 
entire set of the operative parameters of the camera is controllable from a remote 

25 computer (or by the suitable interface unit). The Cameras Module is designed to 
be mountable on diverse surfaces such as a helmet, a moving arm or a static 
boom. In addition to the cameras, the Cameras Module includes the mechanism 
required to change the Inter Pupillary Distance (IPD) and the Convergence Angle 
(DA) between the cameras utilizing at least two electric motor devices such as the 

30 Geared Minimotor made in Japan. 
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Referring now to Fig. 5 where a more detailed view of the components of 
the system of Fig. 2 is shown. Two cameras 22 and 24 are assembled as a camera 
module. Both cameras are equipped with an about XI 2 Zoom lens providing an 
horizontal field of view of about 48.8° to about 4.3" The cameras are controlled 
using serial communication Imks 54 and 56 so that each camera can be separately 
controlled in order to modify the system parameters and to balance the separate 
video channels associated with each camera. The S- Video (Super VHS) video 
signal is transmitted as two components Luminance and Chrominance for the 
forming of a better quality picture. The cameras of the system will preferably 
have an S- Video output. In order to accomplish optimization and balancing of the 
video levels between the cameras the Luminance (y) channel will be used to 
control video signal level. 

Cameras 22 and 24 are mechanically linked by mechanical arm 44 
connected to and controlled by IPD/Convergmg Angle mechanism 46. Camera 22 
transmits video signals via conduit 48 and camera 24 transmits video signals via 
conduit 52 to IPU 50. Control signals corresponding to the various operative 
parameters of camera 22 and camera 24 are transmitted via conduit 54 and 56 
respectively to IPU 50. Control signals resulting from the digital signal 
processing of IPU 50 are sent from IPU 50 to IPD/Convergence Angle control 
mechanism via conduit 58 in order to perform the correct positioning of the 
cameras 22 and 24 in relation to each other. IPU 50 is linked via control conduit 
80 to Operator Controller (OC) 40 and via conduit 82 to HMD 30. HMD 30 
comprises a pair of display units 62 and 64 such as LCD glasses. For the display, 
a known Head Mounted Display could be used such as Sony Japan Model 
LDI-DIOOBE, or the like. The model LDI-DIOOBE is a high-resolution display 
suitable for stereoscopy with the "See-Through" feature. The video signals 
received from IPU 50 are decoded and displayed on display units 62 and 64. In 
the preferred embodunent of the present invention OC 40 is hard-wired to IPU 
50. In another embodiments of the present invention OC 40 may be modified for 

-10- 
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use with an IR (Infra Red) communication controller (like the home TV), by 
voice activation, Radio Transmission, or by any other form of communication 
now present or later developed suitable for this purpose. 

The Cameras Module is connected to an Interface and Processing Unit 
(IPU). The Interface and Processing Unit, which is the "brain" of the system, 
controls the cameras, the mechanism that changes the position of the cameras 
(e.g. IPD and Convergence Angle) and the display. IPU 50 contains several 
functional components. Power control 67 is the main control of the system and 
the only manual control located on IPU 50. Power control 67 is preferably a 
guarded type switch to prevent accidental turn off. The other functional 
components of IPU 50 will be described in association with the following 
drawings. 

Operator Controller unit (OC) 40 connected to PU 50 via conduit 80. OC 
40 is utilized by user 10 to transmit instructions to IPU 50 and to receive from 
IPU 50 the setting-related data of the system and the general status of the system. 
User 10 communicates with the system through a set of controls located on OC 
40. When user 10 initiates an observation/manipulation session the Initialize 
control 68 is activated to evoke a startup-reset procedure in order to set the basic 
parameters of the system such as the initial values of the Field of View, Shutter 
speed, the Focusing distance, the IPD, and the Converging Angle (DA). Readjust 
control 70 is used when the operator wants to make sure that the system is well 
adjusted, or when some environmental parameter such as the ambient lighting 
was altered, Activatmg Readjust 70 will automatically replace some of the 
automatic adjustments normally required to optimize the system in real-tune. 

"See-through" is an advanced feature of known display devices to operate in 
two different modes: a) image display mode, and b) transparent mode. In image 
display mode the screen displays an image generated and transmitted by an 
appropriately connected image-sensing device, while in transparent mode the 
image is removed from the display screen which is then rendered transparent in 
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order to enable an observer to look through the screen at the real scene beyond. 
When activated See-Through control 72 turns the display units 62 and 64 to the 
See-Through mode by rendering the respective display screens transparent 
thereby neutralizing the viewing device and enabling the observer to watch the 
scene through his eyes in the natural line of sight. Another press on See-Through 
control 72 toggles the display devices back to the view provided by the images 
transmitted from cameras 22 and 24 via the respective video channels. When 
display units 62 and 64 are in See-Through mode the rest of the system including 
cameras 22 and 24 continue to function normally. 

The Mode control 74 is proposed to operate cyclically. In order to change 
specific parameters made available to user 10 to modify. Mode control 74 is 
activated repeatedly and intermittently in order to select the appropriate mode and 
associated parameters to be changed. Up control 78 and Down control 76 are used 
to select the value of the respective parameter. For example, during the regular 
operation of the system the default mode is set to Field of View (FOV). In FOV 
mode the operator can control the magnification of the displayed image. By 
activating Up control 78 and Down control 76 the appropriate values a FOV are 
set, the corresponding Zoom-In or Zoom-Out effect is achieved and the image is 
effectively magnified or decreased in size. When the operator wishes to modify a 
different parameter Mode control 74 is activated and the current mode is switched 
to Auto White Balance (AWB). AWB is a method to balance the color 
components of the picture in order to produce natural colors. After switching to 
the desired mode a corresponding indication specifying the new selection will 
appear on Display Screen 66. When AWB mode is selected the activation of Up 
control 78 or Down control 76 will effect suitable selection of preset parameter 
values of AWB such as Indoors, Outdoors, or Auto. Further activation on Mode 
control 74 will effect the selection of the Stabilizers mode and the Up control 78 
and Down control 76 will set the value of the parameter associated with the 
Stabilizers mode such as ON or OFF. Screen 66 could additionally display the 
system status, focus, zoom and various enror messages. 

-12- 
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It would be easily perceived that additional units could be connected to the 
main units described above. For example additional display units of either type 
(additional Head Mounted Displays, Monitors or Video Projectors) could be 
conceivably added to the basic configuration of the system in order to enable 
other members of the crew to observe the images viewed by user 10. The images 
could be also transmitted to outside the operating room via cables or wireless 
means for teaching or consulting purposes. Furthermore the system could utilize a 
communication encoder in order to transmit the video images in a compressed 
format to a remote location for an on-line expert assistance. 

Referring now to Fig. 6 that provides a detailed illustration of the 
components of the Interface and Processing Unit (IPU). IPU 50 comprises a 
processor device 88, a storage device 84, a timing and control unit 86, a power 
supply/DC converter 96, a control unit interface 98, an HMD command interface 
100, a servo interface 90, a camera command interface 92, and a camera video 
interface 94. Storage device 84 of Fig. 6 holds the operative software functions in 
executable binary files. The software functions are computer programs controlled 
by appropriately predetermined and preset parameter values, which are stored in 
suitable data structures. Cameras 22 and 24 transmit signals representing video 
images to camera video interface 94. The signals are transmitted via HMD 
command interface 100 to HMD 30 to be displayed on the respective display 
devices. Cameras 22 and 24 also transmit digital signals representing luminous 
data to processor device 88 to be processed and acted upon if necessary. 
According to the data received fi-om the cameras 22 and 24 processor device 88 
send commands via Camera Command Interface 92 and serial command lines 54 
and 56 to cameras 22 and 24 in order to change the values of parameters in regard 
to the quality of the images such as the focusing distance of cameras 22 and 24. 
Processor device 88 will also send conmiands via servo interface 90 to 
IPD/Convergence Angle mechanism 46 in order to vary appropriately the spatial 
positioning of cameras 22 and 24 in relation to each other thereby establishing an 

-13- 
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optimal IPD and associated Convergence Angle to achieve the best dq)th 
perception for the user while keeping eye fatigue low. 

Operator Ck)ntroller (OC) 40 is connected to Processor device 88. OC 40 
enables user 10 to control specific parameters of the system such as Field of 
View, Focus, Automatic White Balance and the like. The instructions of user 10 
are transmitted to processor unit 88, and translated to specific commands by 
processor device 88 using software functions suitably encoded into executable 
binary format and stored on storage device 84. The commands are transmitted via 
camera command interface 92 and servo interface 90 to cameras 22 and 24 and to 
IPD/Convergence Angle control mechanism 46. 

The cameras 22 and 24 have several automatic built-in functions such as 
Automatic Light Control (ALC) for controlling the video level of a camera by the 
lens iris and the shutter speed. The automatic built-in capabilities utilized to 
correct important operative parameters such as the focusing distance thereby 
enabling processor device 88 to provide for the automatic balancing of the image 
channels in all aspects. Processor 88 sets the respective parameters of the cameras 
22 and 24 after analyzing the received digital control signals. The two images are 
corrected for proper focus, video level, and color balance. Processor 88 also takes 
care of the various tasks at system level such as mterrupt handling, error recovery, 
and the like. 

The performance of the proposed system depends on several parameters 
important for the efficient operation of the system. As the cameras operate in 
close proximity to the object proper focusing distance, and color temperature 
must be maintained. The cameras should be balanced in those aspects in order to 
accomplish uniform performances of both cameras at all times during a given 
session. 

Referring now to Fig. 7 where the logical flow of the main operating 
function of flie system is shown. At step 104 the power is turned on as the 
operator activates Power control 67 of Fig. 5. Subsequently at step 106 the system 

-14- 
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is initialized. The initialization procedure is the system startup during which all 
the functional parameters are reset to predefined values stored in specific data 
structures such as tables and the like on storage device 84. A detailed description 
of the initialization procedure will be set forth hereunder in association with the 
following drawings. At step 108 the OC 40 of Fig. 5 is accessed to read the 
operator's instructions entered manually via specific controls such as pushbuttons 
described in Fig. 5. At step 110 the OC commands are processed by processor 
device 88 of Fig. 6. The detailed description of the process will be set forth 
hereunder in association with the following drawings. At step 112 the average 
video level is read from both cameras and the cameras are balanced regarding 
video levels. The detailed description of the process will be set forth hereunder in 
association with the following drawmgs. At step 114 the focus setting is read 
from both cameras and the cameras are balanced in regard to the focusing 
distance. The detailed description of the process will be set forth hereunder in 
association with the following drawings. At step 116 the IDP and DA are set 
according to previously obtained values of the focus. The detailed description of 
the process will be set forth hereunder in association with the following drawings. 

Referring now to Fig. 8 illustrating the sequence of control activation on 
Operator Controller 40. The basic procedures and modes of operation of the 
system, which are controlled by the operator utilizing OC 40, are as follows: 

a) Initialize Mode 

b) Readjust Mode 

c) Normal run (Automatic mode) 

d) Manual Modification of System Parameters 

Through the OC 40 the operator provided with the option to activate all the 
procedures and operations described above. The Initialize 106 of Fig. 7 procedure 
will work in two situations: when the power is turned on subsequent to the 
activation of Power switch 67 of Fig. 5 or when the Initialize control 68 of Fig, 5 
is being employed. By employing Initialize 68 of Fig. 5 the operator activates a 
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series of predefined fimctions stored on storage devices 84 of Fig. 6 that will be 
executed automatically by processor device 88 of Fig. 6. At step 120 the initialize 
sequence begins starting up the system and subsequently all the functional 
parameters are reset to specifically defined predetermined values. The operative 
stages performed in the Initialize procedure are as follows: 

a) At step 122 the cameras 22 and 24 are turned on 

b) At step 124 the Auto White Balance on cameras 22 and 24 is set to the 
value of Indoor. 

c) At step 126 the Field of View is set to about 35 degrees 

d) At step 128 the Shutter speed is set to about 1/60 seconds 

e) At step 130 the Automatic Gain Control (AGC) is set to OFF. AGC is an 
electronic circuitry that compensates for either audio or video input level 
changes by boosting or lowering incommg signals to match a preset 
level. Usmg AGC, changing input levels can output at a single constant 
setting. 

f) At step 132 the Iris is set to Auto in order to enter Shutter Priority mode 

g) At step 134 the focus is set to the value of about 1.5 meters (about 5 feet) 

h) At step 136 the Inter Pupillary Distance (IDP) is set to the value of about 
75 mm (about 3 inches) and the Convergence Angle to about 2 — 4 
degrees (preferably 2.9 degrees). 

i) At step 138 the Stabilizers are set to ON to prevent automatic changes of 
the current parameters. 

The Initialize function will return control to the main fiinction and 
subsequently enter Normal Run (Automatic Mode) operation m order to begin the 
process of continuously adjusting of the operative system parameters. 

The Readjust procedure 150 is evoked when some environmental 
factors such as the intensity of the ambient lighting has been changed, the 
positioning of the cameras regarding the IDP and the Convergence Angle is 
changing rapidly or both video channels associated with the cameras 22 and 24 
become unbalanced. The steps executed during Readjust procedure 150 are 
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similar to those of Initialize procedure 120 but without resetting all the 
parameters of the system to predefined default values. During Readjust procedure 
150 the See-Through control 72 of Fig. 5 will be preferably activated in order to 
prevent the rapid changes in the images to be displayed to user 10. Subsequent to 
the activation of Readjust control 70 of Fig. 5 the respective function will be 
loaded from storage device 84 and executed by processor device 88 of Fig. 5. The 
steps performed by the function are as following: 

a) At step 152 the Shutter speed is set to about 1/60 seconds 

b) At step 1 54 Automatic Gain Control is set to OFF 

c) At step 1 56 the Iris is set to automatic mode 

d) At step 157 the cameras are balanced respective to video level 

e) At step 158 the focus is set to automatic 

f) At step 159 the cameras are balanced respective to the focusing 
distance. 

The Readjust function will return control to the main logic function 
and as a result will enter Normal Run (Automatic Mode) in order to begin the 
continuous, automatic, and online adjustment of the operative system parameters. 

The operator is provided the option of manually modifying several 
parameters of the system using the Mode control 74 of Fig. 5. By repeatedly and 
intermittently activating Mode control 74 at step 140 the operator selects the 
desired mode of operation in which specific parameters can be modified. All 
modifications will be operative for both cameras. According to the selection made 
a visual indicator specific to the selected mode will appear on display screen 66 
of Fig. 5. The default mode is the FOV mode or the modification of Field of 
View. The FOV is adjusted at step 142 through the activation of Up control 76 or 
Down control 78, which in FOV mode correspond respectively to Zoom-In and 
Zoom-Out operations. As a result changing the value of FOV is effective in 
increasing or decreasmg the size of the images received. Consequently 
magnification of the view is accomplished in this manner. 



-17- 



wo 01/05161 PCT/ILOO/00398 

At Step 144 AWB alternative mode is selected. In AWB mode White 
Balance is adjusted for both cameras. The user provided with the option of 
switching among three predefined values of White Balance Adjustment The 
values are as follows: 

a) Indoor: to correct colors of the images according to artificial 
lighting. 

b) Outdoor: the colors of the images are corrected to the natural 
lighting of about 6500 degrees Kelvin. 

c) Auto - Automatic AWB is performed. 

At step 146 the operator balances the focus between the cameras. The 
focusing distances of both cameras are checked and adjusted if necessary. When 
the focusing distance is modified for the cameras the IPD is calculated and 
modified. Since stereoscopic vision is optimal when the typical average value of 
the IDP is 1/20 of the distance between a stereoscopically enabled observer and 
the nearest clearly seen object of the observed space the value of IPD is set to 
about 1/20 of the focusing distance with a minimum value of about 50 - 60 mm 
(preferably 53 mm). Following the modification of the IDP the Convergence 
Angle DA is obtained by accessing a suitable data structure such as a lookup table 
and extracting the appropriate value. 

Turning now to Fig. 9 that shows the process of automatic balancing 
the video level 112. At step 166 the luminous average video level is read fi-om 
both cameras and at step 168 the values obtained fi-om the cameras is compared to 
each other. At step 170 it is determined whether the difference of the video level 
between the two cameras is greater than a predefined value such as 10%. If the 
difference is greater than the predefined value then step 172 each camera is 
checked to determine which camera has a setting away firom a predefined set 
video level value such as about 1,0 Volt peek to peek. According to the result at 
step 176 or at step 174 the iris of the respective camera is adjusted to the 
predefined set value. Subsequently control returns to step 168 to repeat the 
process of the balancing. When at step 168 it is found that the difference between 
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the video levels of the cameras is less than a predefined value then control returns 
to step 166 to obtain the luminous video level of the cameras again. 



Referring now to Fig. 10 that shows the flow of control associated with 
5 the automatic balancing of the focus setting 114 of the cameras relative to each 
other. At step 180 the focusing distance settings of both cameras are obtained and 
compared for difference at step 182, At step 184 it is determined whether there is 
a difference between the settings. When no difference is found control returns to 
step 180 in order to repeat the balancing process. If the settings are found to be 
10 different at step 186 it is determined which camera has a focus setting that is 
higher than the value of the other camera. According to the result at step 190 or 
step 188 the focus of the camera with a higher focus value is reset to the value of 
the other camera. 

15 Referring now to Fig. 1 1 that shows the flow of logic associated with 

the automatic setting of the IDP and DA 116. At step 194 the focus setting of 
both cameras is read and at step 196 the IDP is calculated. At step 198 the IDP is 
set and at step 202 the value of IPD will enable the extraction of the value of the 
DA fi-om a lookup table held on the storage device at step 200. The value of DA 

20 is set at step 204. The results are fed into the servo interface 90 of Fig. 6, are 
translated into appropriate signals and are sent to the IPD/DA controUmg 
mechanism 46 of Fig. 6. Function control then returns to step 194 to obtain 
readings of the focus settings from both cameras and repeat the process. 

25 Referring now to Fig. 12A, 12B, and 12C showing the mechanical 

structure of Cameras Module 20 of Fig.6 including the motors 216 and 210 with 
associated mechanical parts. Cameras 22 and 24 are hanging down from a base 
218 where the anchoring points of the mechanism to base 218 are bolts holding 
both motors 210 and 216. The front motor 216 is equipped with a double-sided 

30 axis with opposite direction threads. The bolts holding camera 22 and 24 are 
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having the same thread, therefore when motor 216 rotates in one direction the 
front side 212 and 214 of cameras 22 respectively will move closer to each other 
or will move farther from each other. In order to make the described motion 
possible the connection between the bolts and the cameras 22 and 24 housings 
can rotate freely but will not change position. The rear motor 210 is also equipped 
with a threaded shaft that moves both arms 206 and 208. Consequently the 
distance between the rear parts of cameras 22 and 24 will increase or decrease 
according to the direction of the motor 210 movement The described mechanism 
provides the control of both the IPD and the Convergence Angle, 

It would be easily perceived by one skilled in the art that the above 
description merely illustrates the underlying concept of the present invention and 
refers only to one possible arrangement among many conceivable designs that 
could be effective in the manufacture of the proposed system. 

In another embodiment of the present invention auxiliary equipment will be 
added such as recording and playback, audio and communications in order to 
correspond to a typical operation room configuration. 

Yet in another preferred embodiment of the present invention a more 
compact vereion of the system will be provided with an added light source, 
operated on batteries or car battery connector. In this embodiment the system 
could be used by the military or by emergency crews acting outdoors. 

Still in anoAer preferred embodiment of the present invention the proposed 
system will contam monochrome Intensified cameras (ICCD), for Low Light 
operations for military use. 

It will be appreciated by persons skilled in the art that the present 
invention is not limited to what has been particularly shown and described 
hereinabove. Rather the scope of the present invention is defined only by the 
claims which follow. 
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CLAIMS 

I/we claim: 

1. A method of displaying in real time, dynamic, variably magnified, optimized, 
and stereoscopically emulated images of a physical application space to a 
human operator, in order to assist in the process of close observation, and the 
manipulation of objects within said application space, the method comprising: 
generating dynamic images of said application space by image-generating 

devices; 

encoding said dynamic images into encoded electric signals; 

transmitting said encoded electric signals representing the dynamic images to 

display devices positioned in optical alignment with the eyes of said human 

operator; 

decoding the encoded signals representing the dynamic images into 

reproduced images; 
displaying said reproduced images to the eyes of the human operator. 

2. The method of claim 1 further comprises the steps of: 

encoding image quality control parameter values of said image-generating 
devices into digital signals; 

transmitting said digital signals representing said image quality control 

parameter values to a digital signal processing device; 
monitoring the digital signals representing the image quality control parameter 
values of the image-generating devices; 

automatically balancing the image quality control parameter settmgs of the 
image-generating devices; 

automatically adjusting the stereoscopic quality related physical positions of 
the image-generating devices. 
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3. The method of claim 1 further comprises the step of manually controlling the 
image quality control parameters of the image-generating devices. 

4. The method of claim 3 whereby the step of manually controlling the image 
quality control parameters comprises the steps of: 

initializing the image quality control parameter values of the image-generating 
devices; 

readjusting selectively said image quality control parameter values of the 
image-generating devices; 

resetting selectively at least one of the image quality control parameter values 
of the image-generating devices; 

stabilizing the image quality control parameter values of the image-generating 
devices; 

selectively disconnecting the image-generatmg devices from said display 
devices to enable observation of said application space through the display 
devices in the line of sight. 

5. The method of claim 4 whereby the step of resetting selectively comprises the 
steps of; 

controlling the field of view and the magnification of the displayed images; 
regulating the color balance of the displayed images; 
adjusting the sharpness of the displayed images; 
setting the brightness of the displayed images. 

6. The method of claim 2 whereby the step of automatically balancing the image 
quality control comprises the steps of: 

monitoring the image quality control parameter values of a first 
image-generating device; 
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monitoring the image quality control parameter values of a second 
image-generating device; 

comparing said image quality control parameters values of said first 
image-generating device and said second image-generating device; 
equalizing the image quality control parameter values of the first image 
generating device and the second image-generating device; 
repeating the above said steps. 

7. The method of claim 2 whereby the step of automatically adjusting the 
stereoscopic quality comprises the steps of: 

monitoring image quality control parameter values of the image-generating 
devices; 

calculating according to said monitored image quality control parameter 
values the optimal stereoscopic quality control parameter values; 
transmitting said calculated optimal stereoscopic quality control parameter 
values to spatial control devices; 

transforming said calculated optimal stereoscopic quality control parameter 
values into appropriate physical positioning signals; 

physically positioning the unage-generating devices to spatial locations 
optimal for the generation of effective stereoscopic images; and 
repeating the above said steps. 

8. A method of providing a human operator colored, high-definition, variably 
magnified, dynamic, stereoscopic, qualitatively optimized, reproduced video 
images of a physical operating environment in order to aid said human 
operator during an observation and manipulation session associated with 
objects within said environment, the method comprising: 

viewing dynamic images of said observation and manipulation session by 
means of video cameras; 

transforming said dynamic images into encoded video signals; 
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sending said encoded signals representing the video images via a set of 
suitable conduits to respective viewing screens situated between the operating 
environment and the eyes of the human operator; 

decoding the electric signals representing the dynamic images into visually 
reproduced video images; and 

displaying said visually structured video images on said viewing screens. 

9. The method of claim 8 further comprises the steps of: 

transforming image quality control parameter values of the video cameras into 
suitable digital signals; 

sending said digital signals via a set of respective conduits to a specific 
computing device having a digital signal processing capabilites; 
monitoring changes made accidentally, manually or automatically to said 
operative image quality control parameters; 

balancing said image quality control parameter values among the video 
cameras in order to achieve uniform images on said viewing screens; 
automatically optimizing the physical locations of the video cameras in 
respect to each other. 

10. The metiiod of claim 8 further comprises Ae step of manually adjusting flie 
operative image quality control parameter values of the video cameras by the 
human operator. 

11. The method of claim 10 whereby the step of manually adjusting comprises 
the steps of: 

initializing the operative settings of the video cameras to pre-selected values; 
selectively re-adjusting said operative settings of the video cameras to 
pre-selected values; 

selectively resetting at least one operative setting of the video cameras to 
values determined; 
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disabling automatic adjustment of the operative settings of the video cameras; 
disabling the display of video images on the viewing screens to provide clear 
view of the human operator in the line of sight. 

12. The method of claim 11 whereby the step of initializing the operative 
settings comprises the steps of: 

activating the video cameras; 

setting color balance control to predefined value; 

setting magnification to pre-defined value; 

setting shutter speed to pre-defined value; 

disconnecting automatic gain control; 

enabling automatic iris control; 

setting focusing distance to pre-defined value; 

calculating value of inter-pupillary-distance according to focusing distance; 
obtaining value of convergence angle according to value of 
inter-pupillary-distance; 

effecting movements of video cameras correspondmg to 
inter-pupillary-distance and convergence angle; 
stabilizing video cameras controls, 

13. The method of claim 11 whereby the step of selectively re-adjustmg 
comprises the steps of: 

setting shutter speed to pre-defined value; 

disabling automatic gain control; 

enabling automatic iris control; 

enabling automatic focusing control; 

balancing video cameras in respect to luminance level; 

balancing video cameras in respect to focusing distance. 
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14. The method of claim 9 whereby the step of automatically optimizing 
comprises the steps of: 

monitoring the focusing distance settings of the video cameras; 

calculating the optimal value of the inter-pupillary-distance and the 

corresponding convergence angle; 

transforming the resulting values into suitable electric control signals by an 
operative servo motor interface; 

activate appropriate servo motor devices operative in the positioning of the 
video cameras in respect to each other; 
repeating the above said steps. 

15. The method of claim 14 wherein the result of said calculation of the 
inter-pupillary distance is obtained by dividing the focusing distance by a 
predefined value. 

16. The method of claim 14 wherein the result of calculating the converging 
angle is obtained by fetching the desired value from a predefined table 
according to the calculated inter-pupillary distance. 

1 7. A stereoscopic video observation and magnification system comprismg: 
two video cameras mechanically linked, embedded into a cameras module 
wearable by the human operator; 

two display screens embedded into a head mounted display unit wearable by 
the human operator; 

an operator controller unit wearable by the human operator; 
an interface and processing unit; 

a suitable video circuit connecting said two video cameras, said two display 
screens, via which the video cameras transmit suitable electric signals 
representing dynamic images of the operating environment to be displayed on 
the display screens; 
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serial communication links and a serial circuit via which digital signals 
representing the image quality control parameter values of the video cameras 
are sent to the interface and processing unit, and to the operator controller in 
order to enable monitoring and consequently automatic balancing and 
5 optimizing of the reproduced images. 

18. The system of claim 17 the cameras module further comprising: 

a first servo motor coupled to the video cameras to effect positioning of the 
cameras in respect to the distance among the two cameras; 
10 a second servo motor coupled to the two video cameras to effect positioning 
of the cameras in respect to the convergence angle formed by the respective 
lines of sight to the objects in the operating space; 

thereby forming a controllable Inter pupillary distance/converging angle 
mechanism. 

15 

19. The system of claim 17 of the operator controller unit comprising: 

a display screen to indicate the system status, the image quality control 
parameters values, and the current operating mode; 
an initialize control to provide the human operator with the option of setting 
20 the image quality control parameter to pre-defined values; 

a readjust control to provide the operator with the option of re-adjusting 
image quality control parameter values; 

a mode control to manually select image quality control parameters; 
an up control and a down control to manually adjust parameter values 
25 respective to the current mode; and 

a see-through control to disconnect the video signals transmitted by the 
video cameras to the viewing screen in order to enable the operator to view 
the operating space along the natural line of sight 

30 20. The system of claim 19 wherein the controls are pushbuttons. 
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21. The system of claim 17 of the interface and processing unit comprising: 
a main control device to turn on and turn off the system; 
a camera video interface to handle the video signals received from the video 
cameras; 

a camera conmiand interface to handle digital signals corresponding to 
image quality control parameter values received from video cameras and to 
transmit processed digital signals from the processor device to the respective 
cameras; 

a head mounted display interface to transmit the electric signals 
corresponding to the video images to the viewing screens; 
a servo interface to translate video cameras positioning related control 
signals to suitable servo commands and send said servo commands to tiie 
respective location adjustment mechanism; 

a storage device to hold executable control functions, control tables and 
pre-defined image quality control parameter values; 
a processor device to process digital signals respective to the image quality 
control parameter values; 

a control unit interface to receive commands from the operator; 
a power supply/DC converter device; and 
a timing and control unit. 

22. The system of claim 21 wherein the main control device is a guarded switch. 

23. The system of claim 17 wherein the viewing screens are Liquid Crystal 

Devices. 

24. The system of claim 17 wherein the video cameras having S-VHS output. 
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25. The system of claim 17 wherein the viewing screens having built-in 
see-through capability. 

26. The system of claim 17 further comprises recording and playback capabilities 

concerning the generated dynamic images. 
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